Although this Court has long navigated "the complex realm of [employment] testing and test validation," Guardians Ass'n v. Civil Serv. Comm'n of New York, 630 F.2d 79, 82 (2d Cir.1980), once again, we find ourselves in uncharted territory. This case asks us to decide whether the general knowledge test component of New York State's public school teacher certification program is racially discriminatory in violation of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. §§ 2000e-2000e-17. Furthermore, where, as here, the plaintiffs are public employees, the delicate issues of federalism and state police powers come into play in answering the threshold question of which of the defendants, if any, can be considered an "employer" for the purposes of Title VII analysis.

Plaintiffs-appellants, Elsa Gulino, Mayling Ralph, and Peter Wilds, represent a class of African-American and Latino educators in the New York City (the "City") public school system. The essence of their claim is that the New York State Education Department ("SED") and the New York City Board of Education ("BOE") have discriminated against these aspiring teachers through the use of two standardized certification tests, the National Teachers Examination Core Battery ("Core Battery") and the Liberal Arts and Sciences Test ("LAST") of the New York State Teacher Certification Examinations. After a long trial, the district court (Motley, J.**) found in favor of defendants, holding that, although the two certification tests had a disparate impact on the employment prospects of African-American and Latino candidates, defendants avoided Title VII liability because the tests are "job related."

We hold that the district court initially erred in holding that SED is subject to Title VII liability, but we affirm the district court's holding that BOE is subject to Title VII liability as appellants' employer. As to the merits of appellants' Title VII claim, we find both legal and factual errors in the district court's holding in favor of appellees. We are not convinced, however, that the judgment in favor of appellants is warranted as a matter of law. Consequently, we vacate the district court's judgment and remand for further proceedings in accordance with this opinion.

* A

New York's Constitution imposes upon the State a duty to "provide for the maintenance and support of a system of free common schools, wherein all the children of this state may be educated." N.Y. CONST. ART. XI, § 1. This means that it is incumbent upon the State to provide, at a minimum, a "sound basic education" for the State's children. See Bd. of Educ., Levittown Union Free Sch. Dist. v. Nyquist, 57 N.Y.2d 27, 48, 453 N.Y.S.2d 643, 439 N.E.2d 359 (1982); see also Campaign for Fiscal Equity, Inc. v. New York, 100 N.Y.2d 893, 905, 769 N.Y.S.2d 106, 801 N.E.2d 326 (2003) ("[A] sound basic education conveys not merely skills, but skills fashioned to meet a practical goal: meaningful civic participation in contemporary society."). To carry out this command, the State has given the Commissioner of Education broad authority to "prescribe . . . regulations governing the examination and certification of teachers employed in all public schools of the state." N.Y. EDUC. LAW § 3004.1; see also id. §§ 3001, 3002, 3005. Because the State government is, at bottom, responsible for a school's failure to provide a "sound basic education,"1 oversight of the education system at the State level is to be expected.

However, this need for oversight must be balanced against the long-standing principle in New York State that public schools be controlled largely by local school boards. "[Article XI of the New York Constitution] enshrined in the Constitution a state-local partnership in which `people with a community of interest and a tradition of acting together to govern themselves' make the `basic decisions [of] . . . operating their own schools.'" Paynter v. New York, 100 N.Y.2d 434, 442, 765 N.Y.S.2d 819, 797 N.E.2d 1225 (2003) (quoting Levittown, 57 N.Y.2d at 46, 453 N.Y.S.2d 643, 439 N.E.2d 359). Thus, in each school district, the people of the district make "the basic decisions on funding and operating their own schools." Levittown, 57 N.Y.2d at 46, 453 N.Y.S.2d 643, 439 N.E.2d 359 (internal quotation marks omitted); see also N.Y. Civil Liberties Union v. New York, 4 N.Y.3d 175, 181, 791 N.Y.S.2d 507, 824 N.E.2d 947 (2005) (quoting Paynter, 100 N.Y.2d at 442, 765 N.Y.S.2d 819, 797 N.E.2d 1225).2

To insure this emphasis on local control, the New York legislature has, by statute, confided many of the public schools' operating decisions to the discretion of local school boards. For example, school boards approve yearly operating budgets,3 set tax rates, and collect taxes to fund public school budgets.4 See N.Y. REAL PROP. TAX LAW §§ 1302-1342; see also, e.g., N.Y. EDUC. LAW §§ 1608,1716, 1804,1906, 2503, 2554, 2601-a. School boards across the State negotiate labor agreements with unions representing district teachers and other school district employees, see N.Y. CIV. SERV. LAW §§ 200,201.11; N.Y. EDUC. LAW § 3011. This is also the case in New York City, where BOE and the United Federation of Teachers enter into collectively bargained agreements governing the employment of the City's public school teachers. See, e.g., Steven Greenhouse, In Contract Deal, Pay to Rise 10%, City Union Says, N.Y. TIMES, July 14, 2006, at A6; see also, e.g., Agreement Between The Board of Education of the City School District of the City of New York and United Federation of Teachers, Local 2, American Federation of Teachers, AFL-CIO, available at http://www.uft.org/member/contracts/teachers_contract/. Local school boards have also been given broad authorizations to purchase, maintain, and sell property on behalf of a school; to provide transportation to and from schools; to select and provide text books and other learning materials; and generally, "[t]o have in all respects the superintendence, management and control of the educational affairs of the district." N.Y. EDUC. LAW § 1604.30; see also generally §§ 1604, 1709. "Although the New York State Department of Education has substantial responsibility for education in the State, the local school districts maintain significant control over the administration of local school district affairs." Kramer v. Union Free Sch. Dist. No. 15, 395 U.S. 621, 623, 89 S. Ct. 1886, 23 L. Ed. 2d 583 (1969).

Most relevant to this appeal is that local school boards have been given the authority to "employ . . . as many legally qualified teachers as the schools of the district require" and "to determine the rate of compensation of each teacher." N.Y. EDUC. LAW § 1604.8; see also id. §§ 1709.16; 3012.1(a) (prescribing the teacher tenure process to be followed in certain school districts). This means that local school boards are also vested with the authority to enter into contracts with teachers that set forth "the details of the agreement between the parties, and particularly the length of the term of employment, the amount of compensation and the time when such compensation shall be due and payable." Id. § 3011.1. Thus, while some discretion has been taken from local school boards — e.g., the requirement that the teacher be "legally qualified" and that full-time teachers be given an opportunity for tenure — most of the day-to-day management of the affairs of public schools rests entirely with local school boards.5 Nowhere in the State has the principle of "giving citizens direct and meaningful control over the schools," Levittown, 57 N.Y.2d at 46, 453 N.Y.S.2d 643, 439 N.E.2d 359, been more meaningfully practiced than in the City of New York, where, until 1991, the establishment of teacher credentialing requirements fell largely on the BOE, see N.Y. EDUC. LAW §§ 2569-a to 2569-f (1990) (repealed by L.1990, c. 650, § 1, eff. Jan. 1, 1991).

For many years, the BOE was solely responsible for prescribing the credentialing requirements for City teachers. The minimum requirements were a four-year college degree, as well as passage of both a written and an oral examination administered by the City's own Board of Examiners. Once these minimum requirements were met, the candidate was granted a "temporary per diem certificate" (also referred to as an "initial certificate") allowing him or her to teach for one year. At the end of that year, the teacher could either apply for a renewal of the temporary per diem certificate, which would extend the license for one more year, or the teacher could try for a permanent city teaching license.

There were two ways of obtaining a permanent teaching license. First, an applicant could take and pass a second Board of Examiners' test more specific to the candidate's teaching area. The second test had both written and oral components and was designed to test a range of attributes, including an applicant's ability in written English, knowledge of specific subject matter, and ability to apply pedagogical knowledge. The "knowledge" portion of the examination tested content relevant to the teacher's field — i.e., a social studies teacher would not have to answer math questions, etc. Alternatively, an applicant could obtain a permanent teaching license by passing the Core Battery — a general knowledge test intended for aspiring teachers — and then petition for a nomination to a teaching position. Those who took this second route could only be appointed to teach in certain City schools.

Although the City of New York was free to devise its own teacher credentialing programs, its autonomy was limited to some degree because New York City teachers have always had to meet certain state minimum qualifications. A former human resources director for the BOE, Marie DeCanio, testified at trial that, although an "actual [State] certificate was not required[,] . . . comparable background and experience was required" of City certified teachers. Tr. Trans. at 193.

In 1984, the State began requiring that all teachers (except those in New York City and Buffalo) take and pass the Core Battery in order to obtain an initial certificate. Passage of that test soon became a de facto requirement for City teachers as well because State Education law required that the BOE "adopt regulations for the licensure of teachers in the city that [were] substantially equivalent or not less than those required by the state," Tr. Trans. at 1725, and the BOE felt that it could not devise a test on its own that would satisfy the "substantially equivalent" mandate. Those teachers who had passed the first Board of Examiners' test would not have to take the Core Battery. However, there were many teachers with temporary certificates teaching in the City school system, and those teachers could not be permanently certified without passing the Core Battery. Because the City risked decertification of thousands of teachers, SED gave City teachers a five-year period in which to pass the Core Battery. During this period, City teachers could continue to teach with their provisional licenses.

If a teacher did not pass the Core Battery within the five-year period, the teacher's provisional license was revoked. However, a teacher could still obtain a substitute teaching license, and many of the City's teachers who were de-certified at the end of the five-year period remained in their positions as substitutes. Concerned about a sudden glut of full time "substitute" teachers, the State passed a bill that allowed for a test, given by the City's Board of Examiners and only open to substitute teachers in New York City, which would allow the substitutes to obtain an initial certificate without passing the Core Battery. This test, called a "closed examination" because it was closed to all but those who had worked for at least two years as a substitute teacher in the City school system, consisted of only an oral examination. Ms. DeCanio testified that the closed examination "was really a test that was to get people who are working as sub[stitute]s to have an easier method into the system as the initial certificate, and then they would have to complete their [permanent license] requirements after that." Tr. at 239.

In 1987, SED Commissioner, Thomas Sobol, convened a task force to study the State's public schools and make recommendations to improve the system. In March 1988, the Commissioner's Task Force On The Teaching Profession published, "The New York Report: A Blueprint for Learning and Teaching," ("Task Force Report"), which made numerous recommendations for improving the public school system. A large portion of the Task Force Report, titled "Teacher Preparation and Licensure," was dedicated to a discussion of ways to improve the quality of teaching that would be "in keeping with the needs of today's students or teachers." Task Force Report at 614. Specifically, that Report focused on concerns that teachers could be teaching in schools before they obtained their permanent teaching licenses, which could mean that students were being "taught by someone who lacks the necessary subject knowledge or teaching skills." Id.

Two related recommendations in the Task Force Report are relevant to this appeal. First, it recommended that teacher licensing should be done on a "statewide . . . system which will seek to guarantee that before teachers are allowed to work independently in a classroom, they will be thoroughly prepared for that important responsibility." Id. Thus, it was suggested that certification of teachers in New York City and Buffalo follow the same standards as the rest of the State. Second, in order to implement such a system, the Task Force recommended a "sequence leading to professional teacher licensure" that would include a "test of liberal arts knowledge." Id. The Task Force anticipated this test as one of the licensing requirements because of its members' collective belief that "liberal learning is essential to the education of a prospective teacher." Id.

In response to this report, the State passed legislation — effective January 1, 1991 — requiring that all teachers licensed to teach in a public school in New York State be licensed under the same statewide standards. See L.1990, c. 650, § 1 (repealing N.Y. EDUC. LAW §§ 2569 — a to 2569-f); 3004.6 Thus, because the Core Battery was already a prerequisite for non-City public school teachers to obtain their initial certificate, it became a prerequisite for licensure of City teachers, too. However, teachers who did not (or could not) pass the Core Battery could still teach on a temporary license. These temporary licenses could be extended for up to three years, with extensions beyond that granted in exigent circumstances. In the City, where teacher shortages were still acute, many teachers were essentially permanently employed although they held only temporary licenses. Beginning in 1993, SED began to phase out use of the Core Battery and phase in use of the LAST, a test focused on general liberal arts knowledge. During this period, SED also began to urge BOE to push its teachers to satisfy all of the state-mandated licensure requirements, though the State was still granting temporary licenses to City teachers in large numbers to alleviate the City's shortages.

The LAST was developed jointly by SED and National Evaluation Systems ("NES"), a company with considerable experience developing statewide teacher licensing tests. While the Core Battery tested a range of knowledge, skills and abilities that were deemed necessary to being a competent teacher,7 the LAST is used to test only an applicant's knowledge of liberal arts and sciences. It is important to note, however, that the LAST is but one component of the New York State Teacher Certification Examinations ("NYSTCE"). The other requirements for certification include separate assessments of pedagogy and more specific content tests. Currently, those seeking permanent full-time employment as classroom teachers in New York State must now pass the LAST as one of the minimum requirements for attaining an initial certificate. (Converting an initial certificate into a permanent "professional certificate" involves a host of additional requirements not directly relevant to this appeal.)

The LAST itself is a pass-fail examination designed to test basic college-level content. The examination covers five basic areas: (1) "Scientific and Mathematical Processes," (2) "Historical and Social Scientific Awareness," (3) "Artistic Expression and The Humanities," (4) "Basic Communication Skills," (collectively, the "Multiple Choice Sections") and (5) "Written Analysis and Expression" (the "Essay Section"). In order to pass the exam, a test taker must score an average of 200 points on each of the sections. This equates to answering roughly two-thirds of the questions correctly on the Multiple Choice Sections and a score of three points (out of a possible five points) on the Essay Section. However, scoring of the LAST is "compensatory," meaning that a candidate may score below the "passing score" on a certain section and still pass the test if his or her performance on the remaining sections is sufficiently high.

Appellants commenced this action on November 8, 1996, claiming, inter alia, that SED and BOE violated Title VII of the Civil Rights Act of 1964, 42 U.S.C. §§ 2000e-2000e-17. In particular, the complaint alleged that defendants' use of the Core Battery and the LAST to certify public school teachers had a disparate impact on minority job seekers in violation of § 2000e-2(k). Plaintiffs seek back pay, a declaration that the use of the certification tests is unlawful, and an injunction barring further use of the certification tests. Following discovery, both defendants moved for summary judgment on various grounds, including that Title VII does not apply to them. SED argued that Title VII does not apply because it is not plaintiffs' "employer" for the purposes of Title VII. BOE maintained, on the other hand, that it is not liable because it is merely complying with a state law that requires it to de-certify teachers who failed to pass either the Core Battery or the LAST.

The district court denied defendants' motions, finding that both SED and BOE are subject to Title VII liability. See Gulino v. Bd. of Educ. of the City Sch. Dist. of New York, 236 F. Supp. 2d 314 (S.D.N.Y. 2002) ("Gulino I"), modified, 2002 WL 31887733 (S.D.N.Y. Dec.26, 2002). In concluding that SED is potentially liable, the court stated that, in the absence of a direct employment relationship, there are three tests for determining whether a particular defendant is an employer in the Title VII context.8 See id. at 331-32. The district court found that because SED plays a significant role in the management of local school districts, it is subject to liability under the first of the three tests — the so-called "interference" test. See id. at 333. Having reached that decision, the district court declined to discuss the applicability of the other two tests. Id. at 332. The district court also found BOE to be subject to Title VII liability because, although BOE was "merely following the mandates of state law as articulated in the relevant statutes and regulations," id. at 333, Title VII precludes any employer from using discriminatory state law as a defense to liability, id. at 335. Thereafter, SED filed a motion seeking certification for an interlocutory appeal of this determination. The motion was denied by the district court. Gulino v. Bd. of Educ. of the City Sch. Dist. of New York, 234 F. Supp. 2d 324, 325 (S.D.N.Y.2002).

The trial began in December 2002, spanned five months, filled over 3,600 pages of transcript, and included the testimony of 40 witnesses. The district court, applying traditional Title VII burden-shifting analysis, found in favor of the defendants. Gulino v. Bd. of Educ. of the City Sch. Dist. of New York, No. 96-8414, slip. op. at 34 (S.D.N.Y. Sept. 4, 2003) ("Gulino II'"). The court first held that plaintiffs had presented sufficient statistical evidence of disparate impact to make out a prima facie violation of Title VII. Id. at 10-13. Defendants then presented evidence purporting to show that the examinations had been developed and tested to ensure that they were related to the skills and knowledge necessary for public school teachers. At the close of evidence, the court found that the Core Battery was job related and thus, that "the State's certification decisions based on that test was [sic] both valid and reliable." Id. at 15.

As to the LAST, the court found that defendants had failed to establish the "formal validity" of the LAST. Id. at 32; see Guardians, 630 F.2d at 95-105. Notwithstanding that finding, however, the district court — employing a standard it thought was required by the United States Supreme Court in Watson v. Fort Worth Bank & Trust, 487 U.S. 977, 998, 108 S. Ct. 2777, 101 L. Ed. 2d 827 (1988) — held that the LAST is job related "because of the coincidence of three factors: (1) the importance given to the ability to write an essay by those education professionals surveyed by NES; (2) the weight of the essay writing portion of the test; and (3) the fact that the majority of plaintiffs would have passed the LAST but for the essay writing portion." Id. (citations omitted). Having found that the LAST is job related, the district court entered judgment in favor of defendants. Id.

Appellants claim that the district court applied the wrong disparate impact standard, that the district court's factual conclusions regarding the weight of the essay portion of the LAST are clearly erroneous, and that the defendants "misused" the tests in making employment decisions as to "in service" teachers.9 In defense of their victory below, both appellees renew their arguments that Title VII does not apply to them. We address appellees' claims first.

* Title VII makes it "an unlawful employment practice for an employer . . . to fail or refuse to hire or to discharge any individual, or otherwise to discriminate against any individual with respect to his compensation, terms, conditions, or privileges of employment, because of such individual's race, color, religion, sex, or national origin." 42 U.S.C. § 2000e-2(a). Consequently, the existence of an employer-employee relationship is a primary element of Title VII claims. See Tadros v. Coleman, 717 F. Supp. 996, 1004 (S.D.N.Y.1989), aff'd, 898 F.2d 10 (2d Cir.1990) (per curiam); cf. Arbaugh v. Y & H Corp., ___ U.S. ___, 126 S. Ct. 1235, 1245, 163 L. Ed. 2d 1097 (2006) (holding that statutory employee-numerosity requirement is an element of a claim under Title VII). Although Title VII provides definitions of both "employer"10 and "employee,"11 neither definition is particularly helpful in deciding whether an employment relationship exists. In such situations—i.e., where a definition is "nominal" or is otherwise unhelpful—the courts must look outside the statute for guidance. See Nationwide Mut. Ins. Co. v. Darden, 503 U.S. 318, 323, 112 S. Ct. 1344, 117 L. Ed. 2d 581 (1992).

Where a statute "`uses terms that have accumulated settled meaning under . . . the common law, a court must infer, unless the statute otherwise dictates, that Congress means to incorporate the established meaning of these terms.'" Comty. for Creative Non-Violence v. Reid, 490 U.S. 730, 739-40, 109 S. Ct. 2166, 104 L. Ed. 2d 811 (1989) (quoting NLRB v. Amax Coal Co., 453 U.S. 322, 329, 101 S. Ct. 2789, 69 L. Ed. 2d 672 (1981)). In particular, the Supreme Court has stressed that, "when Congress has used the term `employee' without defining it, ... Congress intended to describe the conventional master-servant relationship as understood by common-law agency doctrine." Reid, 490 U.S. at 739-40, 109 S. Ct. 2166; see Clackamas Gastroenterology Assocs., P.C. v. Wells, 538 U.S. 440, 444-45, 123 S. Ct. 1673, 155 L. Ed. 2d 615 (2003) (applying federal common-law of agency to definition of "employee" under ADA); Darden, 503 U.S. at 323, 112 S. Ct. 1344 (ERISA); Reid, 490 U.S. at 743, 109 S. Ct. 2166 (Copyright Act of 1976).

In attempting to give guidance in that regard, the Supreme Court culled the following non-exhaustive, thirteen-factor list of considerations from federal case law and the Restatement (Second) of Agency:

the hiring party's right to control the manner and means by which the product is accomplished .... [;] the skill required; the source of the instrumentalities and tools; the location of the work; the duration of the relationship between the parties; whether the hiring party has the right to assign additional projects to the hired party; the extent of the hired party's discretion over when and how long to work; the method of payment; the hired party's role in hiring and paying assistants; whether the work is part of the regular business of the hiring party; whether the hiring party is in business; the provision of employee benefits; and the tax treatment of the hired party.

Reid, 490 U.S. at 751-52, 109 S. Ct. 2166. Although "[n]o one of these factors is determinative," Reid, 490 U.S. at 752, 109 S. Ct. 2166, "the common-law element of control is the principal guidepost that should be followed," Clackamas, 538 U.S. at 448, 123 S. Ct. 1673;12 see also Eisenberg v. Advance Relocation & Storage, Inc., 237 F.3d 111, 114 (2d Cir.2000); Frankel v. Bally, Inc., 987 F.2d 86, 90 (2d Cir.1993) (citing Spirides v. Reinhardt, 613 F.2d 826, 831 (D.C.Cir.1979)). This Circuit has repeatedly looked to the "Reid factors" in analyzing employment relationships under Title VII. See, e.g., Eisenberg, 237 F.3d at 114; O'Connor v. Davis, 126 F.3d 112, 115 (2d Cir.1997).

In this Circuit, however, "courts turn to common-law principles to analyze the character of an economic relationship `only in situations that plausibly approximate an employment relationship.'" O'Connor, 126 F.3d at 115 (quoting Graves v. Women's Prof'l Rodeo Assoc., 907 F.2d 71, 74 (8th Cir.1990)). Thus, "a prerequisite to considering whether an individual is [an employee] under common-law agency principles is that the individual have been hired in the first instance." Id. at 115 (emphasis added). In determining whether a person has been "hired," we look primarily to "whether [a plaintiff] has received direct or indirect remuneration from the alleged employer." Pietras v. Bd. of Fire Comm'rs of the Farmingville Fire Dist., 180 F.3d 468, 473 (2d Cir.1999) (internal quotation marks omitted). "Where no financial benefit is obtained by the purported employee from the employer, no plausible employment relationship of any sort can be said to exist." O'Connor, 126 F.3d at 115-16 (internal quotation marks omitted).

The district court, in holding that SED is subject to Title VII liability, neglected the common law-based analysis entirely. Instead, as noted above, the district court found that

[t]here are three tests used to determine whether an entity qualifies as an employer in the Title VII context: 1) whether the defendant affirmatively interfered with or affected the plaintiff's direct employment or access to employment opportunities; 2) whether the defendant can be considered a "single" or "joint" employer together with the direct employer under standards borrowed from the National Labor Relations Board; and 3) whether the direct employer is merely an agent or instrumentality of the defendant for the purposes of the discriminatory practice. Plaintiffs assert that SED satisfies all three tests (even though it need satisfy only one).

Gulino I, 236 F.Supp.2d at 331-32 (citations omitted). The court settled on the first of the three tests—the "interference" test—and found that SED's extensive involvement in the affairs of local school districts brought SED within the ambit of the interference test. As discussed more fully below, we disagree with the district court's holding as to the applicability of the interference test, 18 and we reverse. Because we reverse on that issue, we must also address the two other putative tests for employer liability that the district court did not discuss. Having considered the other theories, however, we find that SED is not an employer under any view of Title VII, and we dismiss the Title VII claims against the State defendant.

The "interference" theory of employer liability was first enunciated by the D.C. Circuit in Sibley Memorial Hospital v. Wilson, 488 F.2d 1338 (D.C.Cir.1973),13 but the district court likened the present case to a more recent application of the theory—the Ninth Circuit's decision in Association of Mexican-American Educators v. California, 231 F.3d 572 (9th Cir.2000) (en banc) ("AMAE"). In AMAE, the Ninth Circuit endorsed the test from Sibley, holding that the state of California was subject to Title VII liability because, even though the state was not an employer in the traditional sense, it "interferes" with the employment relationship between public school teachers and the schools that hire them. Id. at 580.

The finding in AMAE of an indirect employment relationship was largely based on "the state's high level of involvement in the operation of local public schools." Id. at 582. The Ninth Circuit noted, "In California, public schools are `a matter of statewide rather than local or municipal concern; their establishment, regulation and operation are covered by the [state] constitution and the state Legislature is given comprehensive powers in relation thereto.'" Id. at 581 (quoting Hall v. City of Taft, 47 Cal. 2d 177, 179, 302 P.2d 574 (1956)). The court also found it significant that "[California]'s involvement is not limited to general legislative oversight but, rather, affects the day-to-day operations of local public schools." Id. Furthermore, the AMAE court found that the state's control over public schools is "plenary" to the point that "individual districts are treated as `state agencies' for purposes of the Eleventh Amendment." Id. at 582 (citing Freeman v. Oakland Unified Sch. Dist., 179 F.3d 846, 846 (9th Cir.1999)).

Judge Motley also briefly mentioned the Second Circuit's own foray into the hazy realm of the interference test: Spirt v. Teachers Ins. and Annuity Ass'n, 691 F.2d 1054 (2d Cir.1982), vacated and remanded on other grounds, 463 U.S. 1223, 103 S. Ct. 3565, 77 L. Ed. 2d 1406 (1983). In that case, defendant TIAA managed a pension fund to which certain employees of Long Island University were required to contribute. The fund made higher retirement payments to male retirees based on the actuarial assumption that women typically live longer than men and would thus draw retirement benefits for a longer period of time. The defendant argued, inter alia, that it should not be held liable under Title VII because it was not the plaintiff's employer. The Court, in finding that the defendant could be liable under Title VII, held that there can be an "indirect" employment relationship where one party "significantly affects access of any individual to employment opportunities." Id. at 1063. Thus, this Court concluded, the defendant's policy of maintaining "sex-distinct mortality tables," id. at 1062, had a disparate impact on female employees, and defendant, as delegee of one of the employer's traditional employment functions, could be held liable under Title VII, id. at 1063.

In concluding that SED is subject to Title VII liability, the district court found that the state's involvement in setting baseline requirements for public schools— including curriculum standards, graduation requirements, and, here, minimum requirements for a teaching license—was sufficient to bring SED within the scope of the interference test. We disagree. In our view, the district court's application of the interference test was wrong for three reasons. First, this Circuit has never adopted a broad reading of the Sibley interference test, and we decline to do so now. The district court's application of the interference test contravenes the plain language of Title VII and strays from basic tenets of statutory interpretation. Second, the district court's wholesale adherence to AMAE was misplaced because the linchpin of the AMAE majority's analysis—i.e., that the State of California has plenary control over the operations of its schools—has no corollary in this case. Third, this Circuit's decision in Spirt does not support the conclusion that the State should be held liable for the disparate impact of the certification tests.

The language of Title VII clearly delineates a liability regime: "It shall be an unlawful employment practice for an employer . . . to fail or refuse to hire or to discharge any individual [on the basis of membership in a protected class]." 42 U.S.C. § 2000e-2(a)(1) (emphasis added). An expansive definition of "employer" contravenes Supreme Court precedent and fundamental canons of statutory interpretation. As Judge Gould's dissent in AMAE aptly states, "A natural reading of [42 U.S.C. § 2000e-2(a)(1)] suggests that the `individual' it references is a potential, current, or past employee of the employer," and that a direct employment relationship "is the typical Title VII case." AMAE, 231 F.3d at 603 (Gould, J., dissenting) (collecting cases).

Beginning with Sibley, however, a number of courts have held that liability under Title VII encompasses not only employers in the traditional sense, but also those who interfere with an individual's access to employment opportunities. See Sibley Mem. Hosp., 488 F.2d at 1341; see also, e.g., AMAE, 231 F.3d at 581; Zaklama v. Mt. Sinai Med. Ctr., 842 F.2d 291, 294 (11th Cir.1988); Doe ex rel. Doe v. St. Joseph's Hosp., 788 F.2d 411, 422 (7th Cir.1986) overruled in part, Alexander v. Rush N. Shore Med. Ctr., 101 F.3d 487, 488 (7th Cir.1996); Armbruster v. Quinn, 711 F.2d 1332, 1336 (6th Cir.1983) abrogated in part on other grounds, Arbaugh v. Y & H Corp., ___ U.S. ___, 126 S. Ct. 1235, 163 L. Ed. 2d 1097 (2006); Shehadeh v. Chesapeake & Potomac Tel. Co., 595 F.2d 711, 722 (D.C.Cir.1978). The Sibley court premised this expansion on its belief that, because "one of Congress' main goals was to provide equal access to the job market," Sibley, 488 F.2d at 1341 (internal quotations omitted), Congress must have impliedly extended liability not only to direct employers but also to those who interfere with a direct employment relationship. We disagree with the Sibley court's expansive approach to interpreting the straightforward language of the statute.

Sibley's central reason for creating "interference" liability under Title VII was that 8 Congress intended a comprehensive solution to employment discrimination and that such a comprehensive solution must entail an expansive view of potentially liable parties. As evidence of this intent, the Sibley court cited the sections of Title VII that extend "interference" liability to employment agencies and labor unions based upon a "highly visible nexus with the creation and continuance of direct employment relationships between third parties."14 Id. at 1342. The court concluded that, because the defendant's "daily operations are of such a character as to have such a nexus," liability was properly extended to that defendant. Id.

This reading of Title VII, however, ignores the very language that Congress employed. Congress did not mention labor unions or employment agencies as examples of additional potentially liable parties under Title VII. Nor did Congress extend Title VII liability in a general way; rather, it limited the statute's additional liability to labor unions and employment agencies. Cf. O'Melveny & Myers v. F.D.I.C., 512 U.S. 79, 86, 114 S. Ct. 2048, 129 L. Ed. 2d 67 (1994) (holding that federal statutory scheme detailing what claims could be made against the FDIC when it is acting as receiver necessarily excluded claims not mentioned in the statute). Congress obviously considered a range of potentially liable parties apart from traditional common-law employers, and in so doing, decided that only these two additional groups would be included within the reach of Title VII. Absent some evidence that Congress intended otherwise, we conclude that all other parties with a similar "nexus" to a plaintiff's employment are excluded from the Title VII liability scheme. See United States v. Smith, 499 U.S. 160, 167, 111 S. Ct. 1180, 113 L. Ed. 2d 134 (1991) ("Where Congress explicitly enumerates certain exceptions ... additional exceptions are not to be implied, in the absence of evidence of a contrary legislative intent."); Greene v. United States, 79 F.3d 1348, 1355 (2d Cir.1996) ("The ancient maxim expressio unius est exclusio alterius (mention of one impliedly excludes others) cautions us against engrafting an additional exception to what is an already complex [statutory scheme].") For the courts "[t]o create additional `federal common-law' [types of liability] is not to `supplement' this scheme, but to alter it." O'Melveny & Myers, 512 U.S. at 87, 114 S. Ct. 2048.

The Sibley court's view of Title VII might also raise additional constitutional concerns. The Supreme Court has warned that a Congressional intent "to alter the usual constitutional balance between the States and the Federal Government" must be "unmistakably clear in the language of the statute." Gregory v. Ashcroft, 501 U.S. 452, 460-61, 111 S. Ct. 2395, 115 L. Ed. 2d 410 (1991) (internal quotations omitted). This requirement safeguards the sovereignty retained by the states in our federal system because it requires Congress to state clearly when and to what it extent "it intends to pre-empt the historic powers of the States." Gregory, 501 U.S. at 461, 111 S. Ct. 2395 (citations omitted). The rule has been employed in a wide range of cases in which federal courts are asked to define the reach of Congressional enactments that threaten to upset the delicate balance of federalism. See, e.g., Pa. Dep't of Corr. v. Yeskey, 524 U.S. 206, 209, 118 S. Ct. 1952, 141 L. Ed. 2d 215 (1998) (whether ADA applies to state prisons); BFP v. Resolution Trust Corp., 511 U.S. 531, 544, 114 S. Ct. 1757, 128 L. Ed. 2d 556 (1994) (whether Bankruptcy Code displaced state laws relating to title in real estate); Gregory, 501 U.S. at 456, 461, 111 S. Ct. 2395 (whether ADEA applies to state law establishing mandatory retirement age for Missouri judges); Hayden v. Pataki, 449 F.3d 305, 323-24 (2d Cir.2006) (en banc) (whether Voting Rights Act applies to state prisoner disenfranchisement laws).

"Providing public schools ranks at the very apex of the function of a State," Wisconsin v. Yoder, 406 U.S. 205, 213, 92 S. Ct. 1526, 32 L. Ed. 2d 15 (1972). The Supreme Court has acknowledged that "[p]ublic education, like the police function, `fulfills a most fundamental obligation of government to its constituency,'" Ambach v. Norwick, 441 U.S. 68, 76, 99 S. Ct. 1589, 60 L. Ed. 2d 49 (1979) (quoting Foley v. Connelie, 435 U.S. 291, 297, 98 S. Ct. 1067, 55 L. Ed. 2d 287 (1978)), and that "public school teachers may be regarded as performing a task `that go[es] to the heart of representative government,'" Ambach, 441 U.S. at 75-76, 99 S. Ct. 1589 (quoting Sugarman v. Dougall, 413 U.S. 634, 647, 93 S. Ct. 2842, 37 L. Ed. 2d 853 (1973)). Thus, there is little doubt that New York acts pursuant to its traditional police powers in performing a core state function—regulating the quality of public school teachers.15 Fortunately, we need not explore the legitimacy of Sibley's view of Title VII and the constitutional concerns raised by the clear statement rule, for we are convinced that the statute does not apply to the State (and thus SED) in the context presented. We see no justification for embracing Sibley's reading of Title VII and its use of the interference test.

2. Factual Differences Between AMAE and This Case

Even if we endorsed Sibley, we still could not affirm the district court's wholesale application of AMAE because of key factual differences between this case and AMAE. The relationship between the State of New York and individual local school districts is very different than the state/local relationship outlined by the Ninth Circuit in AMAE. While the New York Constitution does impose ultimate liability on the State for providing a "sound basic education," the equally important tradition of local control of public schools suggests a measure of autonomy not present in California. Local school districts in the State of New York are not merely "agencies" of the State, and neither the Board of Regents nor the Commissioner of Education exercise plenary control over public schools. Rather, as we have already discussed above, the legislature of New York has specifically delegated much of the public schools' day-to-day operations—including, most importantly here, employment decisions—to the local school districts. While some public school policies come from SED, New York does not have the same "top down" approach to management of the public schools that motivated the Ninth Circuit's decision in AMAE. Thus, even if this Circuit were to follow the legal framework of AMAE, the factual differences between the cases preclude reliance on that decision.16

3. Spirt Does Not Support The District Court's Result

Finally, we think that our decision in Spirt, limited in scope as it is, does not support the decision below. In Spirt, we enunciated a narrow rule based upon a unique factual posture: we held that, where an employer has delegated one of its core duties to a third party—in that case, Long Island University delegated responsibility for the administration of a retirement plan to TIAA—that third party can incur liability under Title VII. In so holding, this Court reasoned that "exempting plans not actually administered by an employer would seriously impair the effectiveness of Title VII." Spirt, 691 F.2d at 1063.

However, in Nationwide Mutual Insurance Co. v. Darden—decided in 1993, 11 years after Spirt—the Supreme Court ruled that the common-law should supply the definition of "employee" in the absence of a statutory definition. See Darden, 503 U.S. at 322, 112 S. Ct. 1344. As we have noted, the holding in Darden "eliminate[d] the chief rationale for employing a broader test in the context of anti-discrimination legislation—namely, that a more liberal construction would better effect the remedial purposes of the ... legislation." Frankel, 987 F.2d at 90. Since Darden, we have declined to adopt as broad a reading of the "interference" test as that advanced by the district court.17 For example, in Kern v. City of Rochester, 93 F.3d 38 (2d Cir.1996), we held that a secretary for the Rochester branch of a firefighters' union was not an employee of the City of Rochester for the purposes of Title VII. Id. at 45. This Court found that Spirt was inapplicable because the union in Kern, unlike the university in Spirt, "ha[d] not delegated any of its employment responsibilities to the [third party]." Id.; cf. Cook v. Arrowsmith Shelburne, Inc., 69 F.3d 1235, 1240 (2d Cir.1995) (holding that Spirt does apply in deciding whether a corporate parent is liable for the Title VII violations of a subsidiary).

Finally, the district court's broad reading of Spirt is doubtful in light of a number of recent decisions by this Court. See Scaglione v. Chappaqua Cent. Sch. Dist., 209 F. Supp. 2d 311, 315 n. 5 (S.D.N.Y.2002) (collecting cases). First, in our 2000 decision in Eisenberg, we held that "a decision on whether a worker is an `employee' [rather than an independent contractor] requires the application of the common law of agency." Eisenberg, 237 F.3d at 113. As noted above, this Court has been careful to point out that "a prerequisite to considering whether an individual is [an employee] . . . is that the individual have been hired in the first instance." O'Connor, 126 F.3d at 115. In Pietras, we noted that evidence of an employment relationship under Title VII will generally turn on "whether [a plaintiff] has received direct or indirect remuneration from the alleged employer." Pietras, 180 F.3d at 473 (internal quotation marks omitted). We reiterated our adherence to the common law agency principles in our decision in York v. Association of the Bar, 286 F.3d 122 (2d Cir.2002), where we found that, "[i]n the absence of a clear contractual employer-employee relationship, a party claiming to be an employee under Title VII must come forward with substantial benefits not merely incidental to the activity performed in order to satisfy this Circuit's remuneration test." Id. at 126. All of these decisions are consistent with the Supreme Court's decisions in Reid and Clackamas Gastroenterology Associates, both of which require adherence to common law principles of agency in Title VII cases, see Reid, 490 U.S. at 740, 109 S. Ct. 2166; Clackamas Gastroenterology Assocs., 538 U.S. at 444, 123 S. Ct. 1673, and a broad application of the interference test is plainly inconsistent with traditional principles of agency law.

Consequently, applying the appropriately narrow reading of Spirt to the facts of this case, we find no basis for holding the State liable. The party here with the clear direct employment relationship, BOE, has not delegated a core employer responsibility to SED. Rather, as discussed above, the State has merely imposed a regulation in exercise of its traditional state concern: teacher competence. Even if we could say that the State has delegated some responsibility to BOE, it would be BOE—the putative third party delegee—that would be liable under Spirt.

4. The Two Remaining Theories of Indirect Employer Liability

As noted above, appellants press two other theories of "indirect" liability—(1) the "single or joint employer" test, originally developed by the National Labor Relations Board, but adopted by the Second Circuit in the Title VII context, see Cook, 69 F.3d at 1240; and (2) the "instrumentality" test.

The "single or joint employer" test utilizes a four-factored analysis developed by the NLRB to determine whether two or more employers can be treated as one for purposes of assigning liability. See id. (identifying four factors as "(1) interrelation of operations, (2) centralized control of labor relations, (3) common management, and (4) common ownership or financial control"). In this Circuit, this analysis has been confined to two corporate contexts: first, where the plaintiff is an employee of a wholly-owned corporate subsidiary;18 and second, where the plaintiff's employment is subcontracted by one employer to another, formally distinct, entity.19 Extending this theory to cases involving the complex relations between levels of government would be impracticable and would implicate the same constitutional concerns that we discussed above in conjunction with the "interference" test. Consequently, we decline to hold SED liable under the "single or joint employer" test.

Appellants' second proffered theory is even less appealing. In advocating adherence to the "instrumentality" theory, appellants cite a footnote in Los Angeles Department of Water & Power v. Manhart, 435 U.S. 702, 718 n. 33, 98 S. Ct. 1370, 55 L. Ed. 2d 657 (1978), which merely states that agents of employers can be held liable under Title VII. This is common sense, and, indeed, Title VII itself explicitly recognizes that "any agent" of an employer will be liable for discriminatory behavior. 42 U.S.C. § 2000e(b). However, appellants do not claim that the BOE is an "agent" of the State. Instead, appellants claim that the State exercises a level of control over the BOE such that the State must be considered an employer under this agency theory. But this is, of course, merely a rehashing of the "interference" and "single employer" tests, and, in any event, it gets the "agency" liability analysis backwards. That is, Title VII liability applies to employers or their agents, and the State is clearly not an agent of BOE. Thus, we find that the so-called "instrumentality" test, to the extent that it is a separate theory of liability, does not apply here.

This leaves us where the district court should have started—at the common law of agency. As detailed earlier, the Supreme Court has given us guidelines for discerning the existence of an employment relationship: traditional indicators of employment under the common law of agency. Under this approach, the State is plainly not plaintiffs' employer for the purposes of Title VII liability.20

As an initial matter, plaintiffs fail to meet the threshold showing that SED hired and compensated them. See O'Connor, 126 F.3d at 115-16; Pietras, 180 F.3d at 473. While SED and the Regents set the baseline qualifications for fully licensed public school teachers, it is BOE who hired, promoted, demoted, and fired teachers in New York City. In fact, as detailed above, employment decisions have been delegated to local school boards by statute in New York. See supra at 6-7. The issues of the teachers' tenure, pay, and benefits are decided by BOE. Finally, while BOE receives some state funds, a portion of which may go to paying teachers, such an indirect source of funds cannot be the basis for Title VII liability.

Even assuming arguendo that plaintiffs could meet that threshold requirement, they would still need to show a traditional master-servant relationship under the multi-factored test from Reid, 490 U.S. at 751-52, 109 S. Ct. 2166. That standard focuses largely on the extent to which the alleged master has "control" over the day-to-day activities of the alleged "servant." The Reid factors countenance a relationship where the level of control is direct, obvious, and concrete, not merely indirect or abstract. For example, under the Reid test, an employment relationship would exist between a partner and an associate of a law firm but not between the National Association of Securities Dealers and a stock broker. Plaintiffs in this case could not establish a master-servant relationship under the Reid test. SED does have some control over New York City school teachers—e.g., it controls basic curriculum and credentialing requirements—but SED does not exercise the workaday supervision necessary to an employment relationship.

Thus, we find that the narrow holding of Spirt does not apply in this case, and we also hold that there is no theory of agency that supports the district court's holding that SED is subject to Title VII liability. Consequently, we vacate that portion of the district court's November 22, 2002 judgment and order that all claims against SED be dismissed.21

Not surprisingly, the district court also found that BOE is an employer subject to Title VII liability. BOE argued in the district court that it should not be held liable for an employment practice required by state law. The court held that "Title VII preempts any state laws in conflict with it," Gulino I, 236 F.Supp.2d at 335 (citations omitted), and that, even though BOE was "merely following the mandates of state law," id. at 333, in using the LAST to certify teachers, it was nevertheless subject to Title VII liability, see id. at 335. BOE does not appear to challenge this finding of the district court; however, we note the district court was correct in holding that the mandates of state law are no defense to Title VII liability. As this court has previously held, "Title VII explicitly relieves employers from any duty to observe a state hiring provision `which purports to require or permit' any discriminatory employment practice." Guardians, 630 F.2d at 105 (quoting 42 U.S.C. § 2000e-7 (1976)). On appeal, BOE appears to change its tack. Instead of arguing lack of discretion, BOE asserts that Title VII does not apply because BOE's use of certification tests for licensing "is a capacity separate and distinct from that of employer under Title VII." BOE Br. at 13. That is, BOE argues that, because it is merely engaged in the licensing of teachers, Title VII does not apply.22

To support its claim, BOE relies almost exclusively on two cases: National Organization of Women v. Waterfront Commission of New York Harbor, 468 F. Supp. 317 (S.D.N.Y.1979) and Lavender-Cabellero v. NYC Department of Consumer Affairs, 458 F. Supp. 213 (S.D.N.Y.1978). BOE argues that these cases exempt the City from all Title VII liability for its licensing activities. Neither case, however, supports BOE's claim. Both cases involve organizations that were merely licensing entities, unlike BOE which acts as plaintiffs' "employer" in the word's ordinary meaning. In National Organization of Women, the Title VII challenge to the activities of the Waterfront Commission failed not only because the Commission was merely a licensing organization, but also because the Commission was created with consent of Congress to manage certain aspects of labor at the New York and New Jersey ports "[t]o loosen the stranglehold on port activities that criminals, racketeers and hoodlums had acquired." Nat'l Org. of Women, 468 F.Supp. at 319 (internal quotations omitted). The district court in that case found that Congress, in passing the bill authorizing the interstate Commission, did not intend to diminish the states' ability to fight crime and corruption by creating new avenues of Title VII liability. Id. at 321. The decision in Lavender-Cabellero is also inapposite to the case at bar. In Lavender-Cabellero, plaintiff challenged the City of New York's system for issuing licenses to process servers. Lavender-Cabellero, 458 F.Supp. at 214. The district court found that the City was merely licensing process servers and not employing them, much as, for example, a state medical board might license, but not employ, doctors. Id. at 215.

While we acknowledge the difficult situation that this creates for the BOE, Title VII requires this result. BOE is not merely a licensing agency; the cases on which BOE relies involve organizations whose raison d'etre is licensing. While SED's interest is in assuring the quality of teachers, it does not hire them. The purpose of BOE, on the other hand, is the education of students in the New York City public schools, and BOE hires and compensates teachers as a means of accomplishing that objective. Consequently, we affirm the district court's decision that BOE is an "employer" subject to liability under Title VII.

What then of the merits of appellants' claim that the district court erred in finding that the LAST is job related and does not violate Title VII? "We review the district court's findings of fact for clear error and its conclusions of law de novo." Wilson v. Nomura Sec. Int'l, Inc., 361 F.3d 86, 89 (2d Cir.2004). After careful review, we find that the district court's decision was premised on both legal and factual error, and we vacate the decision and remand for further proceedings consistent with this opinion.

* Congress has established that it is "an unlawful employment practice for an employer ... to fail or refuse to hire or to discharge any individual, or otherwise to discriminate against any individual with respect to his compensation, terms, conditions, or privileges of employment, because of such individual's race, color, religion, sex, or national origin." 42 U.S.C. § 2000e-2(a). Title VII "proscribes not only overt discrimination but also practices that are fair in form, but discriminatory in operation." Griggs v. Duke Power Co., 401 U.S. 424, 431, 91 S. Ct. 849, 28 L. Ed. 2d 158 (1971). That is, "Congress' primary purpose [in passing Title VII] was the prophylactic one of achieving equality of employment opportunities and removing barriers to such equality." Connecticut v. Teal, 457 U.S. 440, 449, 102 S. Ct. 2525, 73 L. Ed. 2d 130 (1982) (internal quotation marks omitted). Thus, a plaintiff need not show that an employer intended to discriminate to state a claim under Title VII. Rather, "[a] prima facie violation of [Title VII] may be established by statistical evidence showing that an employment practice has the effect of denying the members of one race equal access to employment opportunities." New York City Trans. Auth. v. Beazer, 440 U.S. 568, 584, 99 S. Ct. 1355, 59 L. Ed. 2d 587 (1979); see 42 U.S.C. § 2000e-2(k).

Disparate impact claims follow a three-part analysis involving shifting evidentiary burdens. The Title VII plaintiff bears the initial burden of establishing a prima facie showing of disparate impact. See id. § 2000e-2(k)(1). To do so, the plaintiff must first identify the employment practice allegedly responsible for the disparities, see id. § 2000e-2(k)(1)(B)(i); see also Watson v. Ft. Worth Bank & Trust, 487 U.S. 977, 994, 108 S. Ct. 2777, 101 L. Ed. 2d 827 (1988). The plaintiff must then produce statistical evidence showing that the challenged practice "causes a disparate impact on the basis of race, color, religion, sex, or national origin," 42 U.S.C. § 2000e-2(k)(1)(A)(i). Once the plaintiff has established a prima facie case of disparate impact discrimination, the defendant has two avenues of rebuttal. First, the defendant may directly attack plaintiff's statistical proof by pointing out deficiencies in data or fallacies in the analysis. See Dothard v. Rawlinson, 433 U.S. 321, 331, 97 S. Ct. 2720, 53 L. Ed. 2d 786 (1977); Watson, 487 U.S. at 996, 108 S. Ct. 2777. Second, the defendant may rebut a plaintiff's prima facie showing by "demonstrat[ing] that the challenged practice is job related for the position in question and consistent with business necessity." 42 U.S.C. § 2000e-2(k)(1)(A)(i). Though the terms "business necessity" and "job related" appear to have semantic differences, they have been used interchangeably by the courts. See, e.g., Griggs, 401 U.S. at 431, 91 S. Ct. 849 ("The touchstone [of Title VII analysis] is business necessity. If an employment practice which operates to exclude Negroes cannot be shown to be related to job performance, the practice is prohibited."). Finally, if the defendant meets the burden of showing that the challenged practice is job related, the plaintiff can only prevail by showing that "other tests or selection devices, without a similarly undesirable racial effect, would also serve the employer's legitimate interest in efficient and trustworthy workmanship." Watson, 487 U.S. at 998, 108 S. Ct. 2777 (internal quotations omitted).

The district court found that plaintiffs had made a prima facie showing of disparate impact. Gulino II, No. 96-8414, slip op. at 14. Appellees do not challenge this finding. The district court also found, however, that appellees had shown that both the Core Battery and the LAST were job related, and based on that finding, the district court entered judgment in favor of defendants. Id. at 21, 32, 34. Appellants now challenge only the district court's finding that the LAST was job related.

Though the job relatedness requirement has been defined as a "manifest relationship," Griggs, 401 U.S. at 432, 91 S. Ct. 849, and as "business necessity," id. at 431, 91 S. Ct. 849, the basic rule has always been that "discriminatory tests are impermissible unless shown, by professionally acceptable methods, to be predictive of or significantly correlated with important elements of work behavior which comprise or are relevant to the job or jobs for which candidates are being evaluated." Albemarle Paper, 422 U.S. at 431, 95 S. Ct. 2362 (internal quotation marks omitted). This rule operates as both a limitation and a license for employers: employers have been given explicit permission to use job related tests that have a disparate impact, but those tests must be "demonstrably a reasonable measure of job performance." Id. at 426, 95 S. Ct. 2362.

"The study of employment testing, although it has necessarily been adopted by the law as a result of Title VII and related statutes, is not primarily a legal subject." Guardians, 630 F.2d at 89. Because of the substantive difficulty of test validation, courts must take into account the expertise of test validation professionals. Id. However, judges must also remain aware that reliance upon "the findings of experts in the field of testing" should be tempered by "the scrutiny of reason and the guidance of Congressional intent." Id.

There are two sources of expertise upon which the courts often rely in deciding whether a test has been properly validated. First, a court can look to the testimony of experts in the field of test validation. At trial, both plaintiffs and defendants introduced testimony of experts in the field of educational testing. The district court's extensive findings of fact show that the court relied heavily on these opinions in assessing the job relatedness of the challenged certification tests. Second, despite the usefulness of expert testimony, courts also need clearly established guideposts against which the reliability of the expert testimony can be evaluated. Perhaps the most important source of guidance is the Equal Employment Opportunity Commission's "Uniform Guidelines on Employee Selection Procedures" ("EEOC Guidelines" or the "Guidelines"). See 29 C.F.R. §§ 1607.1-1607.18 (2006). The EEOC Guidelines are intended "to provide a framework for determining the proper use of tests and other selection procedures." Id. at 1607.1(B). Although it has been stressed repeatedly that courts are by no means bound by the EEOC Guidelines, courts rely on them because following the Guidelines promotes consistency in the enforcement of anti-discrimination law. That is, the Guidelines represent "the administrative interpretation of the Act by the enforcing agency" and are thus "entitled to great deference." Griggs, 401 U.S. at 433-34, 91 S. Ct. 849. But the EEOC Guidelines are not only the administrative agency's interpretation of Title VII's requirements for test validation, they also represent the opinion of experts in the field—i.e., the EEOC. Because of that, courts have relied on the EEOC Guidelines since the standards for disparate impact analysis were announced in Griggs. Thus, although we approach the Guidelines with the appropriate mixture of "deference and wariness," Guardians, 630 F.2d at 91, thirty-five years of using these Guidelines makes them the primary yardstick by which we measure defendants' attempt to validate the LAST.

The EEOC Guidelines draw a sharp distinction between tests that measure "content"—i.e., the "knowledges, skills or abilities" required by a job—and tests that purport to measure "constructs"—i.e., the "inferences about mental processes or traits, such as `intelligence, aptitude, personality, commonsense, judgment, leadership and spatial ability.'" Id. at 91-92 (quoting 29 C.F.R. § 1607.14(C)). This distinction, however, goes beyond mere semantics; it has important legal consequences because content validation is generally much easier to achieve than construct validation.23 The result is that "[t]his content-construct distinction ... frequently determines who wins the lawsuit." Id. at 92.

The problem with a sharp distinction is that "knowledges, skills or abilities" can look very much like "constructs." See Id. at 93 (citing research on the hazy distinction between content and constructs). Consistent with a practical approach to test validation, one should not draw too bright a line between content and construct; rather, content and construct represent a continuum that "starts with precise capacities and extends to increasingly abstract ones." Id. at 93. A test will not automatically be subjected to the rigors of construct validation merely because the examination purports to test abstract qualities. Instead, we have stressed that courts should take a "functional approach that focuses on the nature of the job" in analyzing the defendants' test validation procedures. Id. Where a test "attempts to measure general qualities such as intelligence or commonsense, which are no more relevant to the job in question than to any other job, then insistence on the rigorous standards of construct validation is needed." Id. On the other hand, "as long as the abilities that the test attempts to measure are no more abstract than necessary, that is, as long as they are the most observable abilities of significance to the particular job in question, content validation should be available." Id. Common experience tells us that jobs require, and employers should be able to test, a range of abilities, and we must adapt our inquiry to the realities of the testing process.

Guardians laid out a five-part test for use in analyzing the content validity of an employment test:

(1) [T]he test-makers must have conducted a suitable job analysis[;] (2) they must have used reasonable competence in constructing the test itself[;] (3) the content of the test must be related to the content of the job . . . [;] (4) the content of the test must be representative of the content of the job[; and] [there must] be (5) a scoring system that usefully selects from among the applicants those who can better perform the job.

Id. at 95 (footnote omitted). This approach has the advantage of tracking the Guidelines standards while still allowing the courts to take a more functional approach to the analysis. In particular, this test frees the courts from having to draw sharp distinctions between "content" and "construct" or "knowledge" and "ability." Id. at 93-94.

In concluding that the LAST is "job related," the district court declined to apply the standard set forth in Guardians. Gulino II, 96-8414, slip op. at 32-33. Instead the district court found that "the Supreme Court lowered the bar for defendants in disparate impact suits" in Watson v. Fort Worth Bank & Trust, and that, under this allegedly lower standard, the LAST was properly validated. This conclusion was based upon "the coincidence of three factors: (1) the importance given to the ability to write an essay by those education professionals surveyed by NES; (2) the weight of the essay writing portion of the test; and (3) the fact that the majority of plaintiffs would have passed the LAST but for the essay writing portion." Id. (internal citations omitted). This holding was in error. Watson never lowered the bar for defendants; thus, Guardians is still the law in this Circuit. In addition, the court's factual conclusion about the essay portion of the exam is clearly erroneous.

The district court's decision regarding the job relatedness of the LAST was based on an erroneous reading of Watson and a misapprehension of the scope of that precedent. The district court quoted the following passage from Watson in arriving at its conclusion that Guardians is no longer the standard in this Circuit for assessing job relatedness:

Our cases make it clear that employers are not required, even when defending standardized or objective tests, to introduce formal "validation studies" showing that particular criteria predict actual on-the-job performance. In Beazer, for example, the Court considered it obvious that "legitimate employment goals of safety and efficiency" permitted the exclusion of methadone users from employment with the New York City Transit Authority; the Court indicated that the "manifest relationship" test was satisfied even with respect to non-safety-sensitive jobs because those legitimate goals were "significantly served" by the exclusionary rule at issue in that case even though the rule was not required by those goals.

Watson, 487 U.S. at 998, 108 S. Ct. 2777 (quoting Beazer, 440 U.S. at 587 n. 31, 99 S. Ct. 1355). Relying on that passage, the court concluded that, even though the LAST was not formally validated, it was obviously and manifestly related to the legitimate goals of the education department. The district court essentially read Watson as rejecting the need for any formal test validation in Title VII cases.

The district court's legal conclusion regarding Watson raises two concerns. First, it is not clear that the quoted portion of the Watson opinion purported to overrule earlier Supreme Court cases that require employers to conduct validation studies that are at least consistent with the EEOC Guidelines. We think that Watson as a whole is more reasonably read as simply pointing out that some tests measure abilities that are abstract, yet so clearly consistent with legitimate business needs, that formal validation may be either functionally impossible or inadequate as a measure of the test's job relatedness. The examples from Beazer discussed in the quoted passage illustrates a subset of disparate impact cases in which the job relatedness of an employment practice is so patent that formal validation is unnecessary. Neither of those outlying examples are relevant here.

Second, courts should not rely on this portion of Watson because that language comes from a section of the Watson opinion that was joined by only four of the eight participating justices.24 See Watson, 487 U.S. at 1000, 108 S. Ct. 2777 (Blackmun, J. concurring). While that portion of the opinion has persuasive force, this Circuit remains bound by the validation requirements expressed in earlier Supreme Court precedent, namely, Albemarle Paper and Griggs, and as interpreted by Guardians. Further bolstering that conclusion is the fact that this case is much more factually analogous to Albemarle Paper and Griggs than to Watson. Albemarle Paper, Griggs, and Guardians addressed the use of standardized tests in making employment decisions. Watson, on the other hand, addressed the applicability of Title VII to subjective employment practices, such as evaluations by superiors, in making employment decisions. The testing-related cases delineate the appropriate standard for assessing job relatedness.25

In any event, because the BOE fails to point us to record evidence supporting the district court's finding that the LAST was properly validated, we are obliged to vacate the district court's judgment for clear factual error. As we have already noted, the district court based its conclusion regarding job relatedness on "the coincidence of three factors: (1) the importance given to the ability to write an essay by those education professionals surveyed by the NES; (2) the weight of the essay writing portion of the test; and (3) the fact that the majority of plaintiffs would have passed the LAST but for the essay writing portion." Gulino II, 96-8414, slip op. at 33 (internal citations omitted). It is with respect to the last factor that the record is deficient.26

The NES data on test scores is broken down by content and by demographics. The data in the record spans from March 1993 to July 1999. Some background generalizations about scores and race bear mentioning. Between 1993 and 1999, the average pass rate for white test takers ranged from 91% to 94%, while the average pass rate for African American candidates ranged from 51% to 62%, and the average pass rates for Latino candidates ranged from 47% to 55%. All candidates tended to have the highest average scores on the section titled "Artistic Expression and The Humanities," and all candidates tended to have the lowest average score on the section titled "Written Analysis and Expression"—i.e., the "essay" section.

The district court's factual conclusions were clearly erroneous in several key respects. First, in many of the years, African American and Latino candidates in general averaged lower than passing scores on sections other than (and in addition to) the essay section. Thus, in those years, many candidates presumably would have failed the LAST even if they had achieved passing scores on the essay section. Second, in some specific instances involving individual plaintiffs in this case, the essay portion of the test appears not to have determined the outcome as concluded by the district court. On several tries, one of the named plaintiffs failed multiple sections of the exam, such that a passing score on the Essay Section would not have been enough to pass the LAST. On the other hand, two of the plaintiffs failed exams despite having received a passing score on the Essay Section. Finally, the LAST employs a compensatory scoring regime so that a candidate who scores below the passing score on the essay section could still pass the exam by scoring higher on the other sections. Thus, strictly speaking, it is impossible for the test as a whole to be valid simply because one section is job related. That is, because higher performance on one section may compensate for lower performance on another, one particular section could never be the sole reason that a particular group fails the exam. Thus, even under the district court's reading of Watson, on the record presented to us, the job relatedness of one particular section cannot establish the job relatedness of the entire test.

The district court concluded that the LAST could not be properly validated under the Guardians standard at least in part because of a "pervasive lack of documentation." Gulino II, 96-8414, slip op. at 28. Were we to agree, it might be appropriate to direct the entry of judgment on remand in favor of appellants. In fact, however, we are not convinced, as a matter of law, that such judgment is warranted in this case.

It was unquestionably proper for the district court to take into account the fact that NES and SED had either destroyed, or failed to maintain, documents relating to the extensive procedure allegedly employed to validate the LAST. This lamentable failure on the part of NES and SED has considerably complicated an already difficult case.27 But to the extent the district court appears to have concluded that the failure to produce documents was necessarily fatal to formal validation, we disagree.

It is well-established that "[j]ob relatedness cannot be proved through vague and unsubstantiated hearsay." Albemarle Paper, 422 U.S. at 428 n. 23, 95 S. Ct. 2362. Nevertheless, first-hand accounts of those involved in the test validation process, as well as the studied opinions of certified experts, may be sufficient, in some circumstances, to establish the validity of an employment test. This is true even though the EEOC Guidelines view documentation as a necessity if a defendant is seeking to convince a court that a test is "manifestly related" to the employment in question. As we have previously observed, we are not bound to follow the rigid strictures of the EEOC Guidelines "to the letter." Thus, without minimizing the heavy burden that defendant bears in seeking to validate a test without relying on documentary evidence, we cannot say that a lack of corroborating documentation necessarily dooms a defendant's claim of job relatedness.

Because of the identified factual and legal errors, a remand is necessary. On remand, a new judge will have clear guidance as to the legal standard operating in this Circuit. Remand is also appropriate because BOE is now the sole defendant and must tailor its defense to that position.

Consequently, we VACATE the judgment of the district court and REMAND for further proceedings consistent with this opinion. We stress that, on remand, the district court must apply this Circuit's standard for test validation, as described in Guardians and in this opinion. We also urge the court on remand to take into account the structure of the LAST, and how it is scored, when assessing the test's validity. Finally, SED is not properly subject to Title VII liability in this case, and we order that the Title VII claims against defendant SED be dismissed.